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SUMMARY 


The  objective  was  to  determine  pilot  performance  effectiveness  of  color  versus  black-and-white  in  a  wide-screen 
complex  simulator,  in  the  limited  case  of  bombing  and  landing  performance.  With  the  proliferation  of  color  computer¬ 
generated  imagery  (CGI)  systems,  there  has  been  a  trend  toward  the  use  of  color  systems  in  flight  simulation.  Research 
and  development  (R&D)  exploring  the  utility  of  color  in  complex  visual  flight  simulation  was  needed.  The  problem,  in 
brief,  was  to  gain  sufficient  information  on  the  subject  of  color  versus  black-and-white  in  complex  simulation  to  determine 
the  utility  of  color  presentation.  Having  such  information  would  allow  those  in  the  Air  Force  who  procure  simulators 
to  generate  more  realistic  and  cost-effective  specifications. 

The  methodology  here  compared  pilot  performance  using  a  color  visual  presentation  with  performance  using  black- 
and-white.  Bombing  and  landing  activities  were  chosen  because  of  their  high  task  loading  and  dependence  on  visual 
perception.  In  comparing  these  performance  activities  directly,  an  estimate  of  the  effects  of  color  presentation  was 
possible.  The  test  subjects  consisted  of  22  highly  skilled  and  experienced  instructor  pilots,  all  of  whom  had  undergone 
extensive  weapons  delivery  training  and  were  considered  to  be  combat  ready  pilots.  In  the  case  of  bombing,  they  flew 
a  rectangular  bombing  course,  half  of  the  runs  being  in  color  and  half  in  black-and-white.  Circular  bombing  error  was 
determined  on  each  run.  In  the  case  of  landing,  five  different  parameters  were  measured:  (a)  air  speed  at  touchdown, 
lb)  angle  of  attack  at  touchdown,  (c)  vertical  speed  at  touchdown,  (d)  lateral  distance  from  the  centerline  of  the  runway 
at  touchdown  and,  (e)  longitudinal  distance  from  the  ideal  touchdown  point.  Again,  half  the  runs  were  made  in  color 
and  half  in  black-and-white. 

The  findings  were  consistent  throughout  all  test  conditions:  there  were  no  significant  performance  difference  between 
the  color  and  black-and-white  conditions.  Under  the  specific  circumstances  of  this  R&D,  color  does  not  appear  to 
augment  performance.  It  cannot,  of  course,  be  generalized  from  these  findings  that  color  has  no  value  in  complex 
simulations.  More  R&D  will  be  necessary  in  high  fidelity,  high  resolution  complex  color  systems  in  order  to  come  to 
a  valid  understanding  of  the  full  utility  of  color  in  complex  flight  simulation. 

It  is  concluded  that  color  visual  scene  presentation,  within  the  limits  of  this  R&D,  does  not  enhance  performance. 
Based  on  these  findings,  it  is  recommended  that  the  tasks  of  landing  and  range  bomb  delivery  not  be  used  as  justification 
for  color  in  a  flight  simulator  visual  system. 
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PREFACE 


This  research  and  development  (R&D)  represents  a  portion  of  the  R&D  program  of  the  Air  Force  Human 
Resources  Laboratory  Technical  Planning  Objective  3,  the  thrust  of  which  is  aircrew  training.  The  general 
objective  of  this  thrust  is  to  identify  and  demonstrate  cost-effective  training  strategies  and  training  equipment 
capabilities  for  use  in  developing  and  maintaining  the  combat  effectiveness  of  Air  Force  aircrew  members.  More 
specifically,  the  work  was  part  of  the  R&D  program  conducted  under  the  Air  Combat  Training  Research 
subthrust,  which  has  as  its  goal  to  provide  a  technology  base  for  training  high  level  and  quickly  perishable  skills 
in  simulated  combat  environments.  The  purpose  of  this  effort  was  to  compare  pilot  performance  in  bombing 
and  landing  in  the  2B35  TA-4J  flight  simulator,  using  either  a  color  or  a  black-and-white  visual  display.  Mr. 
James  F.  Smith  was  the  project  scientist;  Mr.  Robert  Woodruff  was  the  task  scientist;  Dr.  Robert  S.  Kellogg 
was  the  principal  investigator.  Dr.  Robert  S.  Kennedy  collaborated  in  the  planning  and  analysis  of  the  data. 

The  authors  wish  to  express  appreciation  for  the  helpful  cooperation  of  U.S.  Navy  personnel  at  the  Naval 
Air  Training  Department,  Naval  Air  Station,  Chase  Field,  Texas  and  to  Dr.  David  C.  Hubbard,  the  University 
of  Dayton  Research  Institute,  and  Messrs  Dick  Greatorex,  AFHRL/OT,  and  Tien-Fu  Sun,  Singer  Company, 
who  performed  the  data  analysis. 
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COMPARISON  OF  COLOR  VERSUS  BLACK- AND- WHITE  VISUAL 
DISPLAYS  AS  INDICATED  BY  BOMBING  AND  UNDING  PERFORMANCE 
IN  THE2B35TA-4J  FLIGHTSIMULATOR 


I.  INTRODUCTION 


With  the  proliferation  of  color  computer-generated  imagery  (CGI)  systems,  there  has  been  a  trend  toward  the  use 
of  color  systems  in  flight  simulation.  The  research  with  respect  to  the  need  for  color  in  complex  visual  flight  simulation 
has  been  inconclusive.  Some  studies  show  enhancement  resulting  from  the  use  of  color  in  some  specific  instances,  while 
others  show  none  at  all.  It  seems  clear  that  insufficient  information  is  available  at  present  to  determine  the  utility  of 
color  in  complex  flight  simulation.  Furthermore,  the  answer  is  made  more  complicated  not  only  because  of  the  lack  of 
psychophysical  data  on  the  subject,  but  because  user  acceptance  of  such  color  systems  may  be  a  substantial  factor,  in 
their  effective  use.  The  problem,  in  brief,  is  to  gain  sufficient  information  on  the  subject  of  color  versus  black-and-white 
in  complex  simulation  to  determine  the  utility  of  color  presentation.  Toward  this  end,  this  investigation,  which  involved 
comparisons  of  bombing  scores  and  landing  scores  in  both  color  and  black-and-white  presentations,  was  developed  at 
the  Operations  Training  Division  of  the  Air  Force  Human  Resources  Laboratory  (AFHRL/ OT). 


n.  THE  2B3S  TA-4J  FLIGHT  SIMULATOR 

The  General  Electric  Navy  2B35  light-valve-projected  color  system  was  used  in  the  effort.  This  system  consists 
of  three  full-color  light-valve  systems  which  are  projected  upon  three  back-lighted  screens  that  surround  the  cockpit. 
The  field  of  view  presented  to  the  pilot  is  2 10°  horizontal  and  60°  vertical.  Continuity  of  the  visual  scenes  between  adjacent 
screens  is  maintained  by  video  cursors  and  photo-electric  sensors  which  prevent  image  drift.  The  visual  presentation 
is  repeated  at  the  instructor's  console,  using  three  television  monitors. 

The  heart  of  the  system  is  the  image  processor,  a  high-speed  special-purpose  computer  housed  in  seven  standard 
equipment  cabinets.  A  two-dimensional  view  of  the  simulated  three-dimensional  view  of  the  world  is  computed  from 
a  digital  data  base  that  describes  the  visual  environment.  The  image  is  computed  30  times  per  second  in  the  form  of 
standard  color  television  video  signals  for  512  pictures  elements  along  each  of  485  active  horizontal  raster  lines  for  the 
three  display  channels.  Overall  control  of  the  system,  data  management  functions,  and  initial  simulation  computation 
are  performed  by  a  general-purpose  computation  system  consisting  of  a  PDP-1 1/50  digital  computer  and  related  software, 
interface  equipment,  and  peripheral  devices. 

A  special  computer  program  was  written  to  remove  the  computer  code  for  hues  which  then  essentially  made  the 
system  black-and-white.  This  change  could  be  made  on  the  system  in  less  than  2  minutes  time  from  color  to  black-and- 
white  or  vice  versa.  The  luminous  intensity  of  the  projected  image  was  checked  with  a  sensitive  photometer  and  held 
at  the  same  level  for  both  black-and-white  and  color. 


m.  SUBJECTS 

The  test  subjects  consisted  of  22  highly  skilled  and  experienced  instructor  pilots.  All  of  them  had  undergone 
extensive  weapons  delivery  training  and  were  considered  to  be  combat-ready  pilots.  Their  ages  ranged  from  24  to  39 
years  and  total  flying  time,  from  650  to  9000  hours.  They  were  all  either  Navy  or  Marine  Corps  aviators.  -  9 


IV.  PHASE  I  -  BOMBING 

The  experimental  sequence  of  the  bombing  study  consisted  of  flying  repeated  runs  around  a  30°  dive  bombing 
course.  The  first  run  was  initialized  on  the  downwind  leg  to  the  target,  and  from  then  on,  the  pilot  made  continuous  _• 

rectangular  bombing  patterns,  with  one  forced-choice  bomb  release  made  on  each  run.  The  bomb  target  consisted  of 
the  typical  concentric-circular  target  used  on  bombing  ranges.  Each  pilot  was  briefed  to  fly  repeated  bombing  runs  and 
to  hit  all  of  the  release  parameters  as  closely  as  possible. 
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The  10  pilots  used  in  Phase  1  were  randomly  assigned  to  one  of  two  groups.  Croup  one  flew  10  runs  with  color; 
then  10  runs  with  black-and-white.  Group  two  flew  in  reverse  order.  This  allowed  for  order  effect  differences  when 
analyzing  the  data.  The  time  delay  for  changing  to  color  or  black-and-white  was  never  more  than  2  minutes  in  duration. 

At  the  conclusion  of  the  runs,  each  pilot  was  thoroughly  debriefed  with  respect  to  the  purpose  of  the  study  and 
was  cautioned  not  to  divulge  any  information  concerning  this  study  to  the  other  instructor  pilots. 


V.  PHASE  II -LANDING 

The  experimental  sequence  in  the  landing  study  was  similar  to  the  bombing  study,  except  that  a  simulated  ground- 
based  carrier  landing  was  used  instead  of  a  bombing  run.  A  “footprint”  of  the  carrier  landing  deck  was  displayed  near 
the  threshold  of  a  ground-based  landing  strip,  and  the  pilot  was  instructed  to  land  on  the  simulated  deck,  coming  as 
close  as  possible  to  ideal  landing  parameters.  On  the  first  approach,  the  pilot  was  released  on  downwind  to  the  landing 
area.  Go-arounds  were  not  permitted,  so  the  pilot  was  forced  to  land  on  each  approach.  Also,  the  Fresnel  Lens  Optical 
Landing  System  (“meatball”)  was  not  used,  since  it  would  have  reduced  the  pilot’s  reliance  on  the  general  visual  scene 
content. 

The  sequencing  of  pilots  to  color  and  black-and-white  was  again  randomized  so  that  half  of  the  12  subjects  were 
presented  with  color  first  and  the  reverse  for  the  other  half.  A  series  of  10  landings  was  made  in  one  condition  and  then 
10  more  landings  were  made  in  the  second  condition.  The  parameters  measured  were  as  follows:  (a)  Vj,  airspeed  at 
touchdown;  (b)  V2,  angle  of  attack  at  touchdown;  (c)  V3,  vertical  speed  at  touchdown;  (d)  V4,  X  lateral  distance  from 
the  centerline  of  the  runway  at  touchdown;  and  (5)  Vs,  Y  longitudal  distance  from  the  ideal  touchdown  point.  All  of 
these  parameters  were  typed  out  by  the  2B35  computer  immediately  following  the  landing  pass.  The  pilots  were  instructed 
to  press  for  attainment  of  ideal  landing  parameters  on  each  run. 


VI.  ANALYSIS  AND  RESULTS 

The  statistical  design  for  Phase  1  and  Phase  11  was  a  split-plot  factorial  with  one  between-subjects  or  grouping  factor, 
order  of  presentation  of  visual  systems  (color  first  vs.  black-and-white  first)  and  two  within-subjects  factors,  V  isual  System 
(color  vs.  black-and-white)  and  Trials  (10  landings,  or  bombing  passes,  undereach  color  condition). 

In  Phase  I,  bombing  circular  error  was  used  as  the  dependent  variable.  Ten  pilots  were  randomly  assigned  to  one 
of  two  groups.  The  five  subjects  of  Group  I  flew  10  runs  with  color  and  then  10  runs  with  black-and-white;  Group  2 
flew  in  reverse  order.  As  can  be  seen  from  the  first  ANOVA  table  of  the  appendix,  no  significant  effects  were  discovered 
in  the  Phase  I  analysis,  indicating  no  differences  in  performance  between  color  and  black-and-white.  Table  1  shows 
the  observed  means  for  each  order  and  visual  system  combination. 


Table  1 .  Phase  I  Means  —  Bombing  Circular  Error  Scores 


Order  (Group) 

Visual  System  Black  A  White  First  Color  First 


Color 

Black  and  White 


177.4  ft 
219.7  ft 


245.6  ft 
213.5ft 


In  Phase  II,  the  five  dependent  variables  for  landing  performance  were  airspeed,  angle  of  attack,  vertical  speed, 
x-distance,  and  y-distance.  Twelve  subjects  (six  within  each  group)  participated  in  the  experimentation  in  Phase  II.  Each 
dependent  variable  was  analyzed  separately.  There  were  no  significant  differences  for  performance  in  color  vs.  black-and- 
white  for  these  dependent  variable  main  effects.  The  only  statistically  significant  results  discovered  in  Phase  II  consisted 
of  a  Group  (i.e. ,  order  of  presentation)  by  Visual  System  interaction  for  airspeed  and  Y-distance.  The  three-way 
interaction  was  also  significant  for  Y-distance  (see  last  five  ANOVA  tables  in  the  appendix).  Tables  2  through  6  show 
the  observed  means  for  each  order  and  visual  system  combination. 


Table  2.  Phase  II  Means  —  Airspeed 


Order  (Group) 

Visual  System 

Black  A  White  First 

Color  First 

Color 

130.8  kts 

134.2  kts 

Black  and  White 

132.7  kts 

133.4  kts 

Note.  Optimum  Airspeed  =  125  kts. 


Table  3.  Phase  II  Means  —  Angle  of  Attack 


Order  (Group) 


Visual  System 

Black  A  White  First 

Color  First 

Color 

16.7° 

16.0° 

Black  and  White 

16.5° 

15.9° 

Note.  Optimum 

Angle  of  Attack  =  16.5°. 

Table  4.  Phase  II  Means  —  Vertical  Speed 

Order  (Group) 

Visual  System 

Black  A  White  First 

Color  First 

Color 

524.5ft/min 

620.0ft/min 

Black  and  White 

543.4ft/min 

606.4ft/min 

Note.  Optimum 

Vertical  Speed  =  650ft/min, 

Table  5.  Phase  II  Means  —  X-Distance 

Order  (Group) 

Visual  System 

Black  A  White  First 

Color  First 

Table6.  Phase  II  Means  —  Y- Distance 


Order  (Group) 

- 

Visual  System 

Black  A  White  First 

Color  First 

• 

Color 

12.8  ft 

27.7  ft 

Black  and  White 

71.8  ft 

11.1ft 

vn.  DISCUSSION 

The  results  in  Phase  1  and  Phase  II  show  no  significant  main  effect  differences  in  performance  with  color  as  compared 
with  black-and-white.  This  effort  has  addressed  only  two  specific  sets  of  performance  outcomes  and  cannot,  of  course, 
be  generalized  to  all  flight  performance.  For  example,  the  requirements  for  low  level,  high  speed  flight  may  call  into 
play  greater  use  of  color  for  effective  performance  in  that  complex  arena.  In  addition,  the  limitations  of  resolution  and 
color  fidelity  as  well  as  color  scene  content  in  the  system  studied  may  be  of  importance  as  limiting  factors.  It  would 
seem,  however,  that  the  two  performance  areas  studied,  bombing  and  landing,  are  sufficiently  generic  and  heavily  enough 
task  loaded  to  be  reasonably  representative  procedures  and,  thus,  useful  as  a  part  of  the  overall  evaluation  of  the  utility 
of  color  in  simulation. 

A  corollary  aspect  of  color  in  simulation  is  simply  user  preference.  Most  of  the  pilots  in  thiB  study  preferred  color. 
It  was  just  more  aesthetically  pleasing  to  them.  However,  for  a  few  pilots,  just  the  reverse  was  true;  they  preferred  black- 
and-white.  The  usual  objection  to  color  was  that  it  was  too  “cartoonish”  and  thus  not  realistic  and  served  subjectively 
as  an  invasive  element  in  their  flying  performance.  Clearly,  the  differences  cannot  be  sorted  out  without  the  experimental 
use  of  high  fidelity  color  simulation. 

Aside  from  simply  aesthetic  considerations,  color  in  flight  simulation  would  seem  to  add  to  realism  and  more 
importantly  to  spatial  orientation  and  velocity  vector  information  available  to  the  pilot.  In  general,  the  better  the  spatial 
orientation  and  velocity  vector  information  available  to  the  pilot,  the  better  will  be  the  potential  for  optimal  performance. 
For  this  reason,  it  would  seem  important  to  carry  out  further  studies  in  the  more  advanced  research  flight  simulators 
which  are  capable  of  high  resolution,  high  quality  and  full  spectrum  color,  as  well  as  enriched  and  expanded  scene 
content.  If  such  an  effort  is  then  coordinated  with  transfer  of  training  effectiveness  studies,  a  more  accurate  evaluation 
of  the  value  of  color  in  flight  simulation  can  be  made. 
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APPENDIX:  ANALYSIS  OF  VARIANCE  (ANOVAl  TABLES 
FOR  PHASE  [  AND  PHASE  II  RES  LETS 


PHASE 

I  ANOVA  -  BOMBING 

CIRCULAR  ERROR 

SCORES 

Source 

df 

SS 

MS 

F-value 

p-value 

Group  (G) 

1 

100396.80 

100396. 80 

2.45 

.1541 

Subj  w/in  groups 

8 

328017.37 

41002.17 

Visual  System  (V) 

1 

1315.84 

1315.84 

.10 

.7610 

G  by  V 

1 

1156.81 

1156. PI 

.08 

.7744 

V  by  Subj 

w/in  groups 

8 

109490.87 

13686.36 

Trials  (T) 

9 

45075.80 

5008.42 

.59 

.8627 

G  oy  T 

9 

48563.84 

5395.98 

.63 

.7671 

T  by  Subj 

w/in  groups 

)  /. 

614626.03 

3536.47 

7  by  T 

9 

95671.40 

10630.16 

1.37 

.2184 

1  .ay  V  by  T 

9 

42785.98 

4753.99 

.61 

.  7843 

7  by  T  by  Subj 

w/in  groups 

“'O 

4- 

359664.93 

7773.12 

PHASE  II  ANOVA  -  LANDING  AIRSPEED 


Source 

df 

SS 

MS 

F-value 

p-value 

Group  (G) 

1 

264.60 

264.60 

.85 

.6197 

Subj  w/in  groups 

10 

3099.58 

309. 96 

Visual  System  (V) 

1 

15.00 

15.00 

• 

00 

.5678 

G  by  V 

1 

126.15 

126.15 

5.75 

.0358 

V  by  Subj 

w/in  groups 

10 

219.25 

21.93 

Trials  (T) 

9 

209.68 

23.30 

1.85 

.0690 

G  by  T 

9 

178.90 

19.88 

1.58 

.1325 

T  by  Subj 

w/in  groups 

72 

1131.42 

12.57 

V  by  T 

9 

44.00 

4.89 

.36 

.9514 

G  by  V  by  T 

9 

236.18 

26.24 

1.92 

.0581 

V  by  T  by  Subj 
w/in  groups 


72 


1228.41 


13.65 


PHASE  II  ANOVA  -  ANGLE  OF  ATTACK 


Source 

df 

SS 

MS 

F-value 

p-value 

roup  (G) 

1 

25.35 

25.35 

.85 

.6170 

ubj  w/in  groups 

10 

298.63 

29.86 

isual  System  (V) 

1 

1.35 

1.35 

.40 

.5478 

by  V 

1 

in 

H 

• 

.15 

.04 

.8315 

by  Subj 

w/in  groups 

10 

33.90 

3.39 

rials  (T) 

9 

8.15 

.91 

.80 

.6182 

by  T 

9 

11.98 

1.33 

1.18 

.3192 

by  Subj 

w/in  groups 

90 

101.87 

1.13 

by  T 

9 

15.82 

1.76 

1.62 

.1224 

by  V  by  T 

9 

10.85 

1.21 

1.11 

.3652 

by  T  by  Subj 

w/in  groups 

90 

97.93 

1.09 

15 


PHASE  II  ANOVA  -  VERTICAL  SPEED 


Source 

df 

SS 

MS 

F-value 

p- value 

Group  (G) 

1 

376516.82 

376516.82 

1.52 

.2445 

Subj  w/in  groups 

10 

2474022.00 

247402.20 

Visual  System  (V) 

1 

421.35 

421.35 

.01 

.9279 

G  by  V 

1 

15778.82 

15778.82 

.30 

.6016 

V  by  Subj 

w/in  groups 

10 

527879.75 

52787.97 

Trials  (T) 

9 

381777.98 

42419.78 

1.56 

.1399 

G  by  T 

9 

374880.25 

41653.36 

1.53 

.1492 

T  by  Subj 

w/in  groups 

90 

2450897.90 

27232.20 

V  by  T 

9 

367225.69 

40802.85 

1.32 

.2376 

G  by  V  by  T 

9 

358854.69 

39872.74 

1.29 

.2531 

1  by  T  by  Subj 

w/in  groups 

90 

2783428.80 

30926.99 

I 


PHASE  II  ANOVA  -  X-DI STANCE 


Source 

df 

SS 

MS 

F -value 

p-value 

Group  (G) 

1 

481. 67 

481.67 

1.57 

.2372 

Subj  w/in  groups 

10 

3064.08 

306.41 

Visual  System  (V) 

1 

160.07 

160.07 

1.57 

.2377 

G  by  V 

1 

244.02 

244.02 

2.39 

.1505 

V  by  Subj 

w/in  groups 

10 

1020.22 

102.02 

Trials  (T) 

9 

1633.57 

181. 51 

1.91 

.0594 

G  by  T 

9 

881.42 

97.94 

1.03 

.4211 

T  by  Subj 

w/in  groups 

90 

8537.91 

94.87 

V  by  T 

9 

1398.52 

155.39 

1.77 

.0850 

G  by  V  by  T 

9 

422.07 

46.90 

.53 

.8471 

V  by  T  by  Subj 

w/in  groups 

90 

7911.09 

87.90 

PHASE  II  ANOVA  -  Y-DI STANCE 


Source 

df 

SS 

MS 

F -value 

p-value 

Group  (G) 

1 

26775. 94 

26775. 94 

.29 

.6054 

Subj  w/in  groups 

28 

5052157.05 

22158.58 

Visual  System  (V) 

1 

31258. 83 

31258.83 

1.19 

.3007 

G  by  V 

1 

228598. 53 

228598.53 

8.73 

.0140 

V  by  Subj 

w/in  groups 

10 

261846.91 

26184.69 

Trials  (T) 

9 

290277. 81 

32253.09 

1.96 

.0525 

G  by  T 

9 

283356.06 

31484.01 

1.92 

.0590 

T  by  Subj 

w/in  groups 

90 

1478742.94 

16430.48 

V  by  T 

9 

152802.00 

16978.00 

.82 

.6043 

G  by  V  by  T 

9 

451638.38 

50182.04 

2.41 

.0168 

V  by  T  by  Subj 

w/in  groups 

90 

1873635.59 

20818.17 

18 
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